Skip to content

fix(gate): cancel PR runs only for new commits - #71

Merged
uniqueg merged 2 commits into
mainfrom
fix/70-gate-cancelled
Nov 10, 2026
Merged

uniqueg merged 2 commits into
mainfrom
fix/70-gate-cancelled

Conversation

@uniqueg

@uniqueg uniqueg commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Summary

A newer run on the same commit (for editing the description, marking the pull request ready, reopening it; e.g. Sourcery editing the body) cancelled the CI run in progress. Its gate (passed, "CI passed", if: always()) then failed on the jobs the cancellation stopped, and as the newest check run of a name on a commit counts, that failure blocked the merge (cloud-registry-ui#242: run 42668413208 cancelled by 42668617719 on b183c64; the same happened again today on bb178d3, runs 42816742704/42816969540).

Fix: only a new head commit cancels the runs of a pull request in progress:

cancel-in-progress: ${{ github.event_name == 'pull_request' && github.event.action == 'synchronize' && github.run_attempt == 1 }}

run_attempt == 1: a rerun keeps its run's event, so rerunning an older commit's synchronize run would otherwise cancel the run on the head commit (Sourcery finding). Runs for other actions, and reruns, wait for the run in progress instead, and a waiting run that is replaced never starts a job (no check run at all). A cancelled run then only ever tests an old commit, whose failed gate blocks nothing. The gate keeps if: always() and stays fail-closed: a needed job that failed or was cancelled (timeout, fail-fast, by hand) still fails it.

Not chosen: if: ${{ !cancelled() }} on the gate (suggested in the issue). It skips the gate in cancelled runs, and GitHub counts a skipped required check as passed, so a run cancelled by hand (or replaced by a newer one that hasn't reported yet) would allow the merge without tests. The first version of this branch did that; the adversarial review rejected it.

Changes:

  • .github/workflows/ci.yml: the new cancel-in-progress; the gate keeps always().
  • actions/gate/README.md: the usage example now includes the concurrency block; new section "Cancelled runs" (why always() and not !cancelled(), runs cancelled by hand or by rerunning an older commit's run, the cost: after a non-push event the result comes after up to two runs); the Tests section.
  • tests/gate/test.sh: checks that the gate job of ci.yml and of the README's example runs always() (job-level if: only, with parser self-tests) and that both use the new cancel-in-progress.

No file that callers run changes (actions/gate/action.yml, check.sh are untouched), so per RELEASING.md this needs no release; it appears in the notes of the next one.

Closes #72

Part of #70; the callers follow in #73.

Verification

  • pre-commit run --all-files (all hooks; markdownlint-cli2 v0.24.0 run separately via npx because nodeenv fails locally): passed.
  • bash tests/gate/test.sh, tests/pr-title, tests/labels, tests/publish-release, tests/release: all passed.
  • Mutation check: reverting the gate to ${{ !cancelled() }} or cancel-in-progress to ${{ github.event_name == 'pull_request' }} in ci.yml makes tests/gate/test.sh fail.
  • How GitHub applies concurrency shows only in real runs; the claim that a replaced pending run leaves no check run matches cloud-registry-ui run 42816969540 (cancelled, 0 jobs).
  • CI: all checks passed (Lint, Test, CI passed, CodeQL, Sourcery) on fdbdb10, run 42862438714.
  • Sourcery: 3 threads; 2 fixed in fdbdb10 (reruns cancelling the head's run, test scoped to workflow-level concurrency), 1 answered (a failed gate on an old commit is intended and blocks nothing); all resolved.

Rollout

The callers set cancel-in-progress in their own ci.yml, so a pin bump changes nothing for them: each needs a one-line change of cancel-in-progress to the expression above (their gates already run always()). All pin actions/gate@743e82d… # v1.0.0:

Repo                           PR types                              cancel-in-progress now
Pubgrade                       opened, synchronize, reopened, edited  event_name == 'pull_request'
Pubgrade-Sidecar               opened, synchronize, reopened, edited  event_name == 'pull_request'
TESK                           opened, synchronize, reopened, edited  event_name == 'pull_request'
biohackeu26-ga4gh-sdk-upgrade  default                                event_name == 'pull_request'
cloud-components               opened, synchronize, reopened, edited  event_name == 'pull_request'
cloud-registry                 opened, synchronize, reopened, edited  event_name == 'pull_request'
cloud-registry-ui              default                                event_name == 'pull_request'
docs                           opened, synchronize, reopened, edited  event_name == 'pull_request'
foca                           opened, synchronize, reopened, edited  event_name == 'pull_request'
landing-page                   opened, synchronize, reopened, edited  true (also cancels push runs)
nvflare-wes-testbed            default                                event_name == 'pull_request'
protes                         opened, synchronize, reopened, edited  event_name == 'pull_request'
tesk-api                       opened, synchronize, reopened, edited  event_name == 'pull_request'
tesk-core                      opened, synchronize, reopened, edited  event_name == 'pull_request'
token-exchange-flow-testbed    default                                event_name == 'pull_request'
trs-filer                      opened, synchronize, reopened, edited  event_name == 'pull_request'

cloud-registry and protes (tests/test_workflows.py) and trs-filer (tests/test_config.py) assert the gate's if: always(), which stays valid; their docs/development.md describe the gate. This PR doesn't change any of them.

Declined review findings

  • Quality review: "the awk parser is more complex than what it protects" — kept, but simplified (indent via match, own fixture file, job-level if: only) because a grep can't tell a job-level if: from a step's (adversarial finding, round 1).
  • Adversarial review, round 1, finding 3 (timeout and cancelled()): moot, the gate keeps always().

Open questions

None.

🤖 Generated with Claude Code

https://claude.ai/code/session_011FSoqMpmDCfbACLj3uy6xV

A newer run on the same commit (for editing the description, marking
the pull request ready, reopening it) cancelled the run in progress,
whose gate (`if: always()`) then failed on the jobs the cancellation
stopped. As the newest check run of a name on a commit counts, that
failure blocked the merge (cloud-registry-ui#242).

Let only a new head commit (action `synchronize`) cancel the runs of a
pull request in progress; runs for other actions wait instead, and a
waiting run that is replaced starts no job. A cancelled run then only
ever tests an old commit, whose failed gate blocks nothing. The gate
keeps `always()` and fails closed: `!cancelled()` would skip it in
cancelled runs, and GitHub counts a skipped required check as passed.

Apply this to this repository's CI and the example in the gate's
README, document why, and test that both use the settings.

Closes #70

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011FSoqMpmDCfbACLj3uy6xV
@uniqueg uniqueg self-assigned this Nov 10, 2026
@sourcery-ai

sourcery-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Configuration

Restrict workflow cancellation to first-attempt synchronize pull request runs, while keeping the gate’s always() condition.

  • .github/workflows/ci.yml L22-29, L230

Documentation

Document the concurrency configuration and explain how cancelled runs affect the fail-closed gate and required checks.

  • actions/gate/README.md L40-55, L64-113

Tests

Extend the gate test script to verify the gate condition and workflow-level cancellation settings in CI and the README example.

  • tests/gate/test.sh L34, L68-146

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@uniqueg
uniqueg marked this pull request as ready for review November 10, 2026 17:06
@sourcery-ai

sourcery-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Linked issues

Checked at commit fdbdb10

#72 fix(gate): cancel PR runs only for new commits · 4 of 5 met · 1 to check by hand

# Requirement Status Notes
1 Only a first-attempt pull_request synchronize run cancels runs in progress, configured in this repo's ci.yml. ✅ Met The workflow-level condition checks the event name, synchronize action, and run_attempt == 1.
2 The gate stays if: always() and fails when a needed job fails or is cancelled. ✅ Met The passed job uses always(); the gate tests cover both failed and cancelled needed jobs.
3 Document the concurrency block and cancelled-run behavior in actions/gate/README.md. ✅ Met The README includes the caller configuration example and a Cancelled runs section explaining the behavior and trade-offs.
4 Test the gate and concurrency settings in tests/gate/test.sh. ✅ Met The tests check the gate condition and cancellation expression in both ci.yml and the README example.
5 Verify that same-commit runs do not cancel each other and a new commit still cancels an older run. 👀 Check by hand (runtime) Confirm in real GitHub Actions runs that the concurrency condition produces these cancellation outcomes.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path=".github/workflows/ci.yml" line_range="28" />
<code_context>
 concurrency:
   group: ${{ github.workflow }}-${{ github.event_name == 'push' && github.sha || github.ref }}
-  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' && github.event.action == 'synchronize' }}

 jobs:
</code_context>
<issue_to_address>
**Current head stays merge-blocked**

When an older pull-request `synchronize` run is rerun while CI for a newer head is active, the rerun keeps `github.event.action == 'synchronize'` and shares the PR’s `github.ref` concurrency group, so `cancel-in-progress` cancels the run testing the current head; its `always()` gate fails, leaving the current commit blocked until CI reports again.

Make the cancellation condition distinguish stale `synchronize` runs from the run testing the current head so a stale run cannot cancel it.
</issue_to_address>

### Comment 2
<location path=".github/workflows/ci.yml" line_range="230" />
<code_context>
     runs-on: ubuntu-latest
     timeout-minutes: 5
+    # Not `!cancelled()`: see "Cancelled runs" in actions/gate/README.md
     if: always()
     needs: [lint, test]
     permissions: {}
</code_context>
<issue_to_address>
**Cancelled runs still fail**

The issue asks cancelled runs to leave no failed `CI passed`, but the gate still runs `always()` and fails when a needed job was cancelled. Restricting cancellation to `synchronize` prevents the same-commit case, but a cancelled run can still leave a failed check on the old commit.
</issue_to_address>

### Comment 3
<location path="tests/gate/test.sh" line_range="113" />
<code_context>
+    failures=$((failures + 1))
+  fi
+done
+cancel="$(sed -n 's/^ *cancel-in-progress: *//p' .github/workflows/ci.yml)"
+# The README's example is the same, folded (`>-`) to fit the line length
+cancel_readme="$(awk '
</code_context>
<issue_to_address>
**Misplaced cancellation setting passes**

When the expected expression is placed in a nested, unrelated concurrency block instead of the workflow-level block, the test reads `cancel-in-progress` without checking that it is under the workflow-level `concurrency` key. If the setting is moved into a job-level concurrency block while the workflow-level setting is removed, these extractors still find the expected expression and pass, even though the workflow no longer cancels runs as intended.

Scope both checks to the intended `concurrency` mapping, for example by parsing the YAML structure rather than matching the key anywhere in the file.

Also at `tests/gate/test.sh:115-119`.
</issue_to_address>

Sourcery assessment

Approval pending. 3 findings to address first.

Blocking findings: .github/workflows/ci.yml:28, .github/workflows/ci.yml:230, tests/gate/test.sh:113


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread .github/workflows/ci.yml Resolved
Comment thread .github/workflows/ci.yml Resolved
Comment thread tests/gate/test.sh Resolved
A rerun keeps the event of its run, so rerunning an older commit's
`synchronize` run cancelled the run on the head commit, whose gate then
failed. Cancel only in a run's first attempt; reruns wait instead.

Scope the test of `cancel-in-progress` to the workflow-level
`concurrency` block, and read the README's folded example the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011FSoqMpmDCfbACLj3uy6xV
@uniqueg
uniqueg merged commit 2562d03 into main Nov 10, 2026
7 checks passed
@uniqueg
uniqueg deleted the fix/70-gate-cancelled branch November 10, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(gate): cancel PR runs only for new commits

1 participant